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Abstract 


Benefits from information sharing between an air traffic service provider and 
a major air carrier are evaluated. Aircraft arrival time schedules generated by the 
NASA/FAA Center/TRACON Automation System (CTAS) were provided to the 
American Airlines System Operations Control Center in Fort Worth , Texas, during 
a field trial of a specialized CTAS display. A statistical analysis indicates that the 
CTAS schedules , based on aircraft trajectories predicted from real-time radar and 
weather data , are substantially more accurate than the traditional airline arrival 
time estimates , constructed from flight plans and en route crew updates. The 
improvement offered by CTAS is especially advantageous during periods of heavy 
traffic and substantial terminal area delay ; allowing the airline to avoid large pre- 
dictive errors with serious impact on the efficiency and profitability of flight oper- 
ations. 


1. Introduction 

Research to investigate the benefits of collabora- 
tion in arrival flow management between air traffic 
control service providers and major air carriers is 
ongoing at the NASA Ames Research Center, under 
the sponsorship of the Collaborative Arrival Planning 
(CAP) project. A discussion of the specialized hard- 
ware and software developed for service provider-car- 
rier communication and preliminary results showing 
the benefits of data exchange have been reported by 
Zelenka et al. (ref. 1). This Technical Memorandum 
describes the portion of the CAP effort devoted to 
improving predicted time of arrival accuracy. Specifi- 
cally, the report discusses the sharing of arrival time 
schedules generated by a set of NASA/FAA airspace 
management software tools known as the Cen- 
ter/TRACON Automation System (CTAS). In an 
exploratory project with American Airlines (AAL), 
arrival times predicted by CTAS, using aircraft trajec- 
tory modeling and dynamic airspace constraints, were 
provided to the airline as an alternative to its tradi- 
tional forecasts based on flight plans and en route crew 
updates. It is shown that the CTAS predictions have 
greater accuracy and are potentially valuable for 
improving the efficiency of airline flight operations. 

A need for closer collaboration between airlines 
and air traffic control facilities has arisen from several 
factors related to the increasing complexity of airline 
operations. These include: increased air traffic volume 
and aircraft delays, critical timing requirements in hub 
and spoke operations, and increased air carrier eco- 
nomic pressures. Domestic and world-wide air traffic 
is expected to grow to unprecedented levels over the 


coming two decades: revenue passenger miles world- 
wide of 1.7 trillion in 1996 are anticipated to reach 3 
trillion in 2006, then 4.5 trillion in 2016 (ref. 2). Exist- 
ing demands on the air traffic system routinely exceed 
the capacity of airports, leading to air traffic imposed 
ground and airborne delays of aircraft, estimated to 
cost domestic airlines as much as $3.5 billion per year 
(ref. 3). In the increasingly competitive airline indus- 
try, with its market-driven pricing and very thin profit 
margins, such economic operating penalties are mag- 
nified. 

Of paramount importance in the efficient opera- 
tion and profitability of an airline is adherence to its 
flight schedule. For large air carriers operating in “hub 
and spoke” networks, the maintenance of the carrier’s 
flight schedule at its hubs drives the business effi- 
ciency of the entire flight network, including outlying 
spokes. In a hub and spoke network, many aircraft 
arrive at a central hub airport in rapid succession and 
depart to other spoke cities with very little time at the 
airport gate. This allows an airline to offer service to 
more cities with fewer airplanes. The control of the 
hub arriving blocks or “banks” of aircraft, and their 
subsequent turn-around in departure banks of aircraft, 
is called “bank management.” 

The American Airlines operational control center 
located in Ft. Worth, TX, called System Operations 
Control (SOC), manages over 650 jet aircraft through 
four major hubs, with over 2200 flight segments daily. 
Their largest hub, at Dallas Ft. Worth (DFW) airport, 
operates 58 gates and 9 major arrival/departure banks 
every day. The largest and most complex bank has 
over 60 aircraft arriving within 75 min, then 70 air- 



craft departing in the following 90 min. At maximum 
capacity during this bank, AAL connects over 6000 
passengers, 9000 bags and 30 tons of cargo. 

Reliable and efficient bank management is critical 
to the success of a hub and spoke airline. Closely- 
spaced arrivals and departures in hub and spoke net- 
works arc veiy sensitive to timing miscues, such as 
those caused by bad weather or airport congestion. 
Arrival timing miscues lead to missed passenger and 
crew connections, inefficient ground operations 
caused by incorrect gate assignments, and occasional 
aircraft diversions to alternate airports. These cause 
passenger inconvenience, flight delays and lost airline 
revenue. 

The data generated and provided by the CTAS air 
traffic management tool are directly relevant to an air 
carrier’s hub operations and associated bank manage- 
ment. The system’s highly accurate time of arrival 
predictions, particularly during periods of airborne 
delay and holding, can assist an airline with several 
aspects of fleet operations. These include whether to 
hold spoke flights for a late incoming flight (the “hold- 
go” decision) and whether a flight should be diverted 
to an alternate airport or continue in airborne holding. 
Typically, departing spoke flights are held if an 
inbound feeder flight is expected to be no later than 15 
min from its scheduled arrival time, but released if the 
feeder flight is over 15 min late. Such a holding 
threshold balances the need to maintain the fleet-wide 
scheduling against the desire to maintain individual 
spoke flights. The time separation between the last 
arrival in a bank and the first departure is critical to 
hub operations. Other factors that make specific air- 
craft arrival times critical include crew time limita- 
tions, gate availability, passenger connections and 
ground manpower. Late gate changes cause many 
problems due to special procedures, equipment needs 
and baggage processing demands. The efficiency of 
all such bank management issues is directly tied to the 
accuracy of aircraft time of arrival predictions. 

Under the CAP project, real-time arrival schedul- 
ing and airspace management data supplied by CTAS 
software to en route controllers at the Ft. Worth Air 
Route Traffic Control Center (ARTCC) are being 
shared with airline operations personnel at the AAL 
SOC. The data exchange utilizes a specialized “CAP 
Display System” which captures the arrival schedule 


for AAL flights constructed by CTAS and used by the 
ARTCC controllers. Analysis (ref. 1) has shown that 
CTAS information sharing leads to improved pre- 
dicted time of arrival accuracy, improved strategic 
fleet arrival planning and improved divert/no divert 
decisions when faced with uncertain airborne delays. 
No adverse impact on FAA air traffic control opera- 
tions was found to have resulted from this experimen- 
tal data interchange. 

A brief technical background covering basic air 
traffic control concepts and the arrival time forecast- 
ing methods employed by AAL and CTAS is included 
as section 2 of this document. Sections 3 and 4 
describe numerical and statistical methods for assess- 
ing arrival time accuracy and inferences from a com- 
parison of CTAS and airline arrival time predictions. 
Some concluding remarks appear in section 5. 

2. Background 

2.1 Air IVaffic Control Concepts 

United States airspace is divided into contiguous 
regions known as “Air Route Traffic Control Centers” 
(ARTCCs). These regions, popularly called “Centers,” 
provide air traffic control services to flights which are 
en route to, but still outside of, the immediate vicinity 
of a major airport. In figure 1 , the darkly-shaded area 
shown on the U.S. map, and in enlargement, is the Ft. 
Worth ARTCC. This Center, with an east- west dimen- 
sion of about 400 miles, is further divided into a num- 
ber of subregions known as “sectors.” One or more air 
traffic controllers directs flights traversing a sector to 
assure safe and efficient traffic flow. 

The airspace within about 40 miles of a major air- 
port is known as the “TRACON,” an acronym for Ter- 
minal Radar Approach Control. The map in figure 1 
shows the Dallas Ft. Worth (DFW) TRACON as a 
lightly-shaded region within the Ft. Worth ARTCC. 
TRACON air traffic controllers issue final instructions 
and clearances for arriving and departing aircraft. Air 
traffic entering the DFW TRACON from the Center is 
funneled through four reference points referred to as 
“comerposts ” A typical traffic pattern is illustrated in 
figure 2(a) by a set of aircraft “tracks” representing all 
AAL flights landing at DFW on April 22, 1998, 
between 4:30 PM and 6:30 PM CDT. These tracks are 
two-dimensional (X,Y) projections of three-dimen- 
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Figure 1 . U.S. Air Route Traffic Control Centers. 

sional aircraft trajectories constructed from radar posi- 
tion measurements. The tracks are seen to converge 
onto the comerposts, marked as filled circles, and then 
to follow more restricted routes within the TRACON. 
Just distinguishable at the center of the diagram are the 
track-traced outlines of individual runways. 

During periods of heavy traffic, known as rushes, 
the comerposts serve as “metering fixes.” Time-based 
metering is a procedure for constraining traffic enter- 
ing the TRACON so that the rate of flow does not 
exceed the capacity or impact the safety of airport 
operations. To meet the constraints of metering, con- 
trollers may need to delay some aircraft. This may be 
done with speed reductions, temporary heading 
changes (vectoring) or holding patterns. Figure 2(b) 
illustrates aircraft tracks during a rush period with sub- 
stantial controller-initiated delay. Many of these tracks 
show evidence of delay by vectoring, and a few also 
show indications of “fix balancing,” a procedure 
whereby aircraft were reassigned from the overloaded 
top-left metering fix to the alternate fix at the bottom- 
left. 

It is important to realize that the delay added dur- 
ing metering is not easily predicted by airlines that tra- 
ditionally rely on arrival times estimated from flight 
plans and infrequent crew updates, and that the conse- 
quent uncertainty in arrival times may lead to disrup- 


tion of the closely orchestrated pattern of hub and 
spoke operations. 

2.2 Airline Time of Arrival Predictions 

Major airlines estimate their flight arrival times by 
tracking time deviations from the nominal flight plans. 
The flight plan created for the flight crew by an air- 
line’s dispatcher includes estimated times for “out” of 
the airport terminal gate, take “off’ from the ground, 
crossings of several navigational fixes en route, land- 
ing “on” the ground and parking “in” the destination 
airport terminal gate. The OUT/OFF/ON/IN times, 
commonly referred to as “OOOI” times, are critical to 
airline operations, as they determine whether an air- 
line is running on or off schedule, and if off-schedule, 
to what degree. An airline too far off schedule begins 
to misconnect flights and suffer operational and eco- 
nomic losses. 

The flight plan, once updated to reflect the actual 
takeoff time, is generally a good approximation of the 
en route or cruise portion of a flight. However, any 
unexpected re-routing or delays can significantly alter 
the flight’s estimated landing or “ON” time. American 
Airlines’ flight procedures call for the flight crew to 
update its time of arrival at “changeover,” defined as 
nominally 20 to 30 min from landing. A typical 
sequence of OFF, changeover and ON times is illus- 
trated by the schematic in figure 3. Although vari- 
able, flight crews typically provide an arrival time 
update once they receive the destination airport’s 
Automated Terminal Information Service (ATIS). As 
ATIS is provided through a line-of-sight transmitter, 
such updating varies with the geography of the desti- 
nation airport. At DFW, such changeover estimates 
are provided approximately 30-35 min from landing. 
These updated estimated ON times, called E/ON 
times, thus reflect any en route delays the flight has 
encountered. At AAL, the updated E/ON is broadcast 
back to the SOC through the Aircraft Addressing and 
Reporting System (ACARS). 

Even if no en route deviations occur to alter the 
nominal flight plan navigational fix times, a flight is 
typically subject to terminal area flight plan deviations 
(e.g., speed decreases, vectoring) that significantly 
alter its ON time. Note that terminal area delays begin 
well outside the TRACON, typically starting approxi- 
mately 30 min from landing at DFW. Such ATC ter- 
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Figure 2(a). Aircraft tracks in Ft. Worth ARTCC and DFW TRACON during a typical rush period from 4:30 to 
6:30 PM on April 22, 1998. 
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Figure 2(b). Aircraft tracks in Ft. Worth ARTCC and DFW TRACON during a heavy-traffic rush period from 
10:00 AM to noon on April 14, 1998. 
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Figure 3. Time markers used by American Airlines. 


minal area deviations differ with each day and arrival 
rush and are difficult for airlines to predict. 

2.3 CTAS Time of Arrival Predictions 

The Center TRACON Automation System 
(CTAS) was developed by the FAA and NASA to 
facilitate the work of air traffic service providers. 
CTAS is a suite of decision support tools that provides 
computer-generated advisories to assist both en route 
and terminal area controllers in efficiently managing 
arrival traffic. CTAS “build 2" consists of two sets of 
tools, the Traffic Management Advisor (TMA) and the 
passive Final Approach Spacing Tool (pFAST) (ref. 
4). TMA generates runway assignments, landing 
sequences and landing times for arriving aircraft. A 
principal product of TMA is a sequence list, with any 
necessary per-aircrafl mclenng delay for arriving air- 
craft entering TRACON airspace. Passive FAST pro- 
vides updated runway and sequence advisories for 
arrival traffic within the TRACON. TMA and pFAST 
may be operated in concert or independently; when 
operated together their demonstrated benefits are addi- 
tive. 

The CTAS scheduling advisories are constructed 
from detailed aircraft trajectories, with input from 
flight plans, aircraft performance models, and real- 
time radar and weather data. Using these trajectories 
and controller-input airspace capacity and operational 
constraints, arrival times are predicted at several refer- 
ence points, including the metering fixes and runway 


thresholds. The present analysis uses only the pro- 
jected times for reaching the runway threshold, as 
given by the CTAS/TMA tool. The threshold crossing 
time, just seconds from touchdown, is a close approxi- 
mation to the “wheels down” landing time recorded by 
the airline. This time marker is hereafter referred to 
simply as the “arrival time.” For each incoming flight, 
CTAS provides two continually-updated projected 
arrival times. The “estimated time of arrival” (ETA) is 
calculated under the assumption that no further delay 
will be imposed before landing. The “scheduled time 
of arrival” (STA) is the ETA plus any delay that 
CTAS recommends to satisfy constraints of safety and 
capacity. CTAS maintains an optimal schedule of 
arrivals, subject to considerations of first-come first- 
serve order, minimum safe separation, and airport and 
runway throughput capacities. 

The CTAS/TMA ETAs and ST As are calculated 
every 12 sec and recorded every 60 sec for subsequent 
data analysis. ETA values are updated continually 
from current radar and weather data until an aircraft 
has landed. STAs are updated until an aircraft is about 
19 min from its metering fix (generally, about 30 min 
from landing), at which time they are “frozen” to pro- 
vide stable values for controller use. The CTAS 
sequence lists and recommended per aircraft delay are 
provided to the ARTCC Traffic Management Coordi- 
nator (TMC) through a video display and to the sector 
controllers through their radar displays. The TMC’s 
display includes timelines for traffic scheduled to the 
metering fixes and runways. A portion of a typical dis- 
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play pattern is illustrated in figure 4, where time is 
shown in the center column, with current time at the 
bottom. Aircraft are represented in the left-hand col- 
umn by tags ordered according to the indicated 
ETAs.The right-hand column indicates the aircraft 
STAs, together with the delay needed to meet the 
STA. The times and aircraft tags move toward the bot- 
tom of the display as time advances. 

Under CAP, the CTAS schedule timelines are dis- 
played in the AAL SOC at a terminal adjacent to one 
of the airline’s strategic flight operations positions. 
The aircraft tags of all non-AAL aircraft are de-identi- 
Fied to prevent a situation of perceived competitive 
advantage. This CAP display is the means by which 
CTAS scheduling information is shared with the air- 
line (ref. 1 ). 

3. Analysis Methods 

3.1 Raw Data 

Basic diagnostic quantities for analyzing arrival 
time predictions were extracted from data recorded by 
the CTAS/TM A tool and from data recorded by Amer- 
ican Airlines. These data were collected in April 1998, 
when the CAP Display System began continuous 
operational use at the AAL SOC. The raw data include 
actual arrival times of AAL flights at the DFW airport 
and arrival time estimates made prior to landing by 
both the airline and by CTAS. The AAL database pro- 
vided the actual ON time for each landing flight, 
together with the E/ON estimate at takeoff, the 
changeover time, and the E/ON estimate at 
changeover. The CTAS data provided time-dependent 
ETA and STA values, recorded every 60 sec. 

Figure 5 illustrates the rate of traffic flow into 
DFW during one of the days, April 14, on which data 
were collected. The horizontal axis is the time of day, 
and the vertical axis shows the number of aircraft 
landing within 10 min intervals. It is evident that there 
are distinct periods of heavy traffic — the “rush” peri- 
ods. Markings at the bottom of the plot indicate four 
rushes selected for analysis, starting 10:00 AM, noon, 
4:30 PM and 7:30 PM. Arrival time data for these 
rushes were extracted from CTAS and AAL record- 
ings made on nine different days (April 13-17 and 
April 21-24), giving a total of 36 separate analysis 
periods. The traffic conditions during these times var- 



Figure 4. CTAS timeline display. 


ied from relatively light, unmetered flow to heavy 
flow with metering at all approach fixes. The air traf- 
fic flow configuration at DFW during the analysis 
periods is described in table 1. Listed are the flow 
direction (north/south) and the airport acceptance rate 
(AAR), which is the number of aircraft that the TRA- 
CON can accept from the ARTCC per hour. During 
rushes with low AAR and high traffic volume, flights 
were subject to increased spacing and delay. 

3.2 Derived Data 

In the present analysis, the accuracy of a flight’s 
predicted arrival time is measured as the difference 
between the actual and predicted landing times. With 
this prescription, the accuracy of each airline arrival 
time prediction is determined as the quantity “ON time 
minus E/ON time,” using either the E/ON reported at 
takeoff or at changeover. The accuracy of each CTAS 
prediction is determined as the airline-supplied ON 
time minus either the CTAS ETA or the STA. 

Because the arrival time predictions are time- 
dependent, recorded at takeoff and at changeover by 
the airline and every 60 sec by CTAS, conventions 
must be established for the times at which airline and 
CTAS predictions are compared. Accordingly, the 
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time (CDT) 


Figure 5. AAL traffic flow into DFW airport on April 14, 1998. 


analysis uses the concept of a “horizon time” for each 
aircraft, defined as the continuously varying interval 
between the local time and the final landing time. An 
aircraft’s horizon time approaches zero as it reaches 
the runway. Arrival time accuracy was calculated at 
three standard horizon times: 1 5, 30 and 60 min before 
landing. The accuracy measure for the airline’s arrival 
time predictions (ON-E/ON) is based on the last E/ON 
reported before the horizon time, i.e., the E/ON at 
takeoff if the horizon time is before changeover, or the 
E/ON at changeover if the horizon time is at or after 
changeover. Similarly, the accuracy of the CTAS pre- 
dictions (ON-ETA and ON-STA) uses the ETA or 
STA at the recording time closest to the horizon time. 

With the above conventions, an accuracy measure 
was assigned to each arrival time predicted by the air- 
line and by CTAS during the selected rush periods. 
These data provide the basis for the analytical studies 
described in the following sections. 


3.3 Visualization and Interpretation Tools 

Graphical and statistical methods were used to 
uncover trends and draw conclusions from the large 
quantity of diagnostic data. The fundamental visual- 
ization tool employed was a histogram plot showing 
the distribution of arrival time prediction accuracy 
during one rush period. Representative examples of 
this distribution are displayed in figure 6. 

The data plotted in figure 6 refer to AAL flights 
arriving at DFW during a 2 hour rush fieri od starting 
4:30 PM on April 22, 1998. A track diagram depicting 
the trajectories of these flights in Center and TRA- 
CON airspace is given in figure 2(a). In figure 6, the 
top histogram was derived from AAL-recorded data 
and the bottom histogram from CTAS data. On each 
of these plots, the horizontal axis represents the accu- 
racy of predicted arrival times (actual minus predicted 
time), and the vertical axis shows the number of air- 
craft with prediction accuracy within 1 min intervals. 
The measure of accuracy was taken as ON-E/ON for 
the airline predictions and ON-STA for the CTAS pre- 
dictions. Positive values correspond to aircraft that 
arrived later than predicted and negative values to air- 


7 




Table 1 . Air Traffic Configuration at DFW Airport 


Period 

AAR, Flow Direction 

13 April 99 

1000 - 1200 CDT 

120, South 


1200- 1400 

120, South 


1630- 1830 

108, South 


1930-2130 

120, South 

14 April 

1000- 1200 

114, South 


1200- 1400 

114, South 


1630- 1830 

120, South 


1930-2130 

120, South 

15 April 

1000 - 1200 

1 14, South 


1200- 1400 

1 14, South 


1630- 1830 

120, South 


1930-2130 

120, South 

16 April 

1000- 1200 

120, North 


1200- 1400 

120, North 


1630- 1830 

120, North 


1930-2130 

120, North 

17 April 

1000- 1200 

126, North 


1200- 1400 

126, North 


1630- 1830 

126, North 


1930-2130 

126, North 

21 April 

1000- 1200 

126, North 


1200- 1400 

126, North 


1630- 1830 

126, North 


1930-2130 

126, North 

22 April 

1000- 1200 

126, North 


1200- 1400 

126, North 


1630- 1830 

126, North 


1930-2130 

126, North 

23 April 

1000- 1200 

120, South 


1200- 1400 

120, South 


1630- 1830 

120, South 


1930-2130 

1 20, South 

24 April 

1000- 1200 

120, South 


1200- 1400 

1 20, South 


1630- 1830 

1 20, South 


1930-2130 

1 20, South 
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Figure 6. AAL and CTAS time of arrival prediction accuracy at the 30 minute prediction horizon during the 4:30-6:30 PM 
rush on April 22, 1998. 


craft that arrived early. For instance, the top plot indi- 
cates that three aircraft arrived 10 min later than the 
airline predictions. Because the horizon time for this 
example, 30 min from landing, is also the average time 
at which changeover E/ON estimates were reported, 
about half of the airline’s arrival time predictions were 
based on the changeover E/ON and about half on the 
E/ON at takeoff. As an aid in visualizing the shapes 
of the data distributions, each histogram is overdrawn 
with a Gaussian curve fit to the mean and standard 
deviation of the prediction accuracy values. 

To further assist in interpreting and comparing 
data sets, the numerical values of several statistically- 
based properties are shown on each histogram plot. 
These include the mean value of the arrival time pre- 
diction accuracy and three quantities representing the 
width of the accuracy distribution: the standard devia- 
tion with respect to the mean, and the fractions of 
flights arriving within ± 2 min and within ± 5 min of 
their predicted times. 


In the final stage of the analysis, airline and CTAS 
predictions for all 36 rush periods were summarized 
and compared. Section 4.3 describes this procedure in 
detail. 

4. Analysis Results 

4.1 Typical Rush Period 

The data in figure 6 illustrate a rush period with 
moderate traffic and minimal delay. Each histogram is 
an accuracy distribution for a set of 57 AAL flights 
which landed during the 2 hour period beginning 
4:30 PM on April 22, 1998, and had CTAS-recorded 
STAs at the prediction horizon 30 min prior to land- 
ing. It is seen that the mean arrival time accuracy of 
these aircraft is 0.3 min late in the airline predictions 
and 0.7 min early in the CTAS predictions. A mean 
accuracy within 1 min is typical for the rushes studied, 
occurring in 75% of the CTAS predictions and 70% of 
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predictions by the airline. 

The observation that the mean per- aircraft arrival 
time error is generally small and comparable in the air- 
line and CTAS predictions suggests that this statistic, 
by itself, is not a meaningful measure of arrival time 
prediction accuracy. The mean prediction accuracy 
may be especially misleading if large positive and 
negative values cancel in the averaging. That this 
indeed occurs for the rush shown in figure 6 is evi- 
denced by the larger width of the distribution repre- 
senting the airline predictions. A better indicator of the 
spread in the data from aircraft arriving significantly 
early or late could thus be a statistic measuring the 
mean absolute value of the prediction accuracy. One 
such statistic is the standard deviation with respect to 
the mean, shown on the plots with the entry labeled 
“sigma.” Because sigma is derived from the mean 
square of the accuracy values, it is especially sensitive 
to the largest values present. By this measure, the 
CTAS arrival time predictions (sigma = 1 .9 min) are 
seen to offer substantial improvement over predictions 
by the airline (sigma = 5.0 min). 

An alternative measure of the spread in the predic- 
tion accuracy is provided by entries on each histogram 
plot giving the fraction of arrival times predicted to a 
specified accuracy, either within ± 2 min or within 
± 5 min of the actual arrival time. For the typical rush 
of figure 6, it is seen that the airline predictions are 
accurate to within ± 2 min for 46% of the flights, but 
that CTAS achieves this accuracy for a significantly 
larger 68% of all flights. Moreover, accuracy within 
±5 min was achieved in 88% of the airline predic- 
tions, but in all 100% of the predictions by CTAS. 

Figure 6 also shows that the CTAS data contain 
fewer outlier points representing prediction anomalies 
large enough to disrupt flight operations. It has already 
been noted that three flights arrived 10 min later than 
the airline predictions, but it is even more significant 
that one flight arrived 24 min earlier. With such uncer- 
tainty in the arrival time, the airline was likely unpre- 
pared with a gate for the flight. In contrast, no flight 
arrived more than 5 min earlier or 4 min later than the 
times predicted by CTAS. 

Additional analyses have shown that the improve- 
ment in arrival time accuracy offered by CTAS at the 
30 min horizon of the “typical” rush period of figure 6 


extends to other rushes and prediction times. The his- 
togram plots in figure 7 compare airline and CTAS 
arrival time predictions for the typical rush at horizon 
times 15, 30 and 60 min before landing. The diagrams 
in the center for the 30 min horizon duplicate figure 6. 
The 60 min horizon is generally close to the time when 
an aircraft is acquired by center radar and first appears 
in the CTAS recording. Consequently, some aircraft 
are not yet recorded by CTAS at this horizon time, and 
fewer data points are plotted. The 30 min horizon is 
close to the time when the CTAS STA is frozen, and 
the 15 min horizon is approximately the time when 
aircraft enter the TRACON airspace, where controllers 
rely on pFAST rather than TMA delay advisories. It is 
evident that the airline’s arrival time predictions are 
similar at the 60 and 30 min horizons, as measured by 
the accuracy distribution widths and the number of 
outlier points. These predictions improve slightly by 
the 15 min horizon time, when most changeover esti- 
mates have been received. The improvement is evi- 
denced by a 40% reduction in the standard deviation 
parameter. In contrast, CTAS predictions improve 
between the 60 and 30 min horizons but remain nearly 
constant inside of the 30 min horizon, where STA val- 
ues are frozen. More significantly, however, it is seen 
that the CTAS predictions are more accurate at all 
horizon times. The widths of CTAS accuracy distribu- 
tions are 50-70% of corresponding airline distribution 
widths, and, whereas the airline distributions contain 
outlier points with 10-24 min errors at each horizon 
time, there is only one CTAS outlier. This occurred at 
the 60 min horizon, when an arrival time forecast from 
an initial flight-plan-based STA was inaccurate by 19 
min. Upon receipt of radar positional data, CTAS 
improved the aircraft’s STA, and, by the 30 min hori- 
zon, it no longer appeared as an outlying point. 

A similar set of accuracy distribution histograms 
in figure 8 demonstrates that CTAS arrival time accu- 
racy at small prediction horizons may be improved by 
using ETAs in place of STAs. These plots refer to the 
same rush period and prediction horizons used for fig- 
ure 7. The top row of plots showing airline prediction 
accuracy is identical in the two figures, but the bottom 
row is changed in figure 8 to show CTAS accuracy 
measured as ON-ETA. It should be recalled that ETAs 
are calculated under the assumption that aircraft will 
have no further delay, and that the updating of ETAs 
continues beyond the time at which STAs are frozen. 
Thus, while the CTAS STAs are generally preferred in 
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Figure 7. CTAS STA-based versus AAL time of arrival prediction accuracy during the 4:30-6:30 PM rush on April 22, 1998. 




92 02 91 OL S 0 92 02 91- 01 9 0 



92 02 91 01 9 0 92 02 SL 01 9 0 



92 02 91- OL 9 0 92 02 91 01 9 0 

S|Baujv jo jaqtunN sibaujv jo jaqiunN 


12 


Figure 8. CTAS ETA-based versus AAL time of arrival prediction accuracy during the 4:30-6:30 PM rush on April 22, 1998. 


AAL Prediction Accuracy 



C T- 

^ N- 

1 d ? 

c £ £ 

o EE 
o c\j in 
5 < < 
u_ + + 





92 02 91 


to 

CM 



01 9 0 




9Z 02 SL 0L 9 0 

s|eaujv jo jeqwriN 


c 

£ (DO) 

•goo 
c £ £ 

OEE 
o cm tn 
2 < < 
LL + + 



p E 


c g 
co E 
0 o> 
2 ( 7 ) 


in 

CM 


LO 


in' 


c 

E, 

< 

I— 

i i 
2 
inO 


uo 

CM 


92 02 SI 01 9 0 

S|baujv jo jaqwriN 


13 


Figure. 9. CTAS ETA/STA-based versus AAL time of arrival prediction accuracy during the 10:00-noon rush on April 14, 1998. 


forecasting arrival times (especially during rushes 
with significant delay), the ETA may become the bet- 
ter predictor inside the “freeze horizon” at about 30 
min before landing. Moreover, the ETA continues to 
improve and may become a highly accurate predictor 
as aircraft approach landing and all necessary delay 
has been absorbed. These trends are evident in figure 7 
as a steepening and narrowing of the ETA accuracy 
distribution with decreasing prediction horizon. It is 
seen that 93% of ETAs are accurate to within 2 min at 
the 15 min horizon, whereas, from figure 6, only 66% 
of the STAs have this precision. In this example, the 
ETAs also show a distinct, although less dramatic, 
advantage at the 30 min horizon, but this advantage 
may not hold during periods of larger delay. There- 
fore, further accuracy comparisons in this report are 
based on ETAs at the 15 min prediction horizon and 
on STAs at all larger horizon times. 

4.2 Large-delay Rush Period 

Figure 9 illustrates arrival time accuracy for a rush 
with particularly heavy traffic and large delay, cover- 
ing the period from 10:00 to noon CDT on April 14, 
1998. This is the period for which aircraft tracks were 
shown in figure 2(b). The top row of histogram plots 
again shows the accuracy of the airline’s arrival time 
predictions and the bottom row the accuracy of CTAS 
predictions. The format deviates slightly from previ- 
ous figures in that, for the reasons given in section 4.1, 
CTAS predictions are based on the aircraft STAs at 
the 60 and 30 min horizons and on ETAs at the 15 min 
horizon. 

The airline has more difficulty than CTAS in fore- 
casting this high-traffic rush, as evidenced by the 
larger widths of its accuracy distributions and by the 
larger number flights arriving considerably early or 
late. If flights with arrival times deviating by 10 or 
more minutes from the airlines latest prediction are 
designated as “outliers,” then the airline accuracy dis- 
tribution is seen to have 10 outliers at the 60 min hori- 
zon, 8 at 30 min, and 5 at 15 min. Most of these flights 
arrive late— one as much as 25 min late. The CTAS 
predictions, on the other hand, show in total only 2 
outliers, both at the 60 min horizon. At the 15 min 
horizon, CTAS predicts all arrivals to within 6 min. 

Arrival time uncertainty can seriously disrupt an 
airline’s strategic planning and operations efficiency. 


For example, in the instance of the aircraft arriving an 
unexpected 25 min late, the airline likely held a con- 
necting flight that it should have released. The incor- 
rect decision to hold might then have initiated a chain 
of schedule disruptions propagating throughout the 
system. 

4.3 Statistical Summary for All Rush Periods 

The analysis results presented in previous sections 
refer to a single rush period, either the “typical” rush 
or the sample “high-delay” rush. In this section, results 
for all rushes studied are presented as a series of dia- 
grams in summary format, beginning in figure 10 with 
diagrams summarizing the uncertainty in arrival time 
predictive accuracy as measured by the widths of 
accuracy distributions. The data in this figure were 
derived from 72 histogram distributions representing 
airline and CTAS arrival time accuracy at the 30 min 
horizon during 36 rush periods (4 rushes per day on 9 
days). The standard deviation of the accuracy distribu- 
tions is plotted as a function of the time of day at mid- 
rush, i.e., CDT one hour after the start of the two-hour 
rush period. The 18 points at each rush time refer to 
CTAS and airline predictions on each of the 9 days, 
with the airline data represented by triangles and the 
CTAS data by squares. Open symbols refer to the indi- 
vidual rush periods and closed symbols to averages 
over the 9 days. 

At each rush time shown in figure 10, the standard 
deviation points for airline accuracy distributions lie 
generally above the corresponding CTAS points, with 
an average factor of 2 separation. This distinction 
between airline and CTAS predictions is especially 
evident for the two rush periods on April 14 indicated 
with arrows. The period centered at 11:00 CDT is the 
high-traffic, large-delay rush for which difficulties in 
the airline’s arrival time forecast were discussed in 
section 4.2. Difficulties with the second period at 
20:30 CDT on April 14 cannot be easily explained by 
the traffic volume or delay, both of which are average 
for the rushes at this time. During these two periods, 
the standard deviations for the airline accuracy distri- 
butions are abnormally large (7.2 and 7.3 min), 
whereas much smaller widths for the corresponding 
CTAS distributions (2.7 and 1.9 min) indicate that 
CTAS experienced no exceptional predictive diffi- 
culty. Finally, it should be noted, from the small varia- 
tion in the solid-point averages, that the improvement 
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Figure 10. Standard deviation of AAL and CTAS arrival time accuracy distributions for all rush periods. 


in predictive accuracy offered by CTAS is consistent 
over rush times throughout the day. 

The conclusion that the CTAS advantage seen in 
figure 10 extends to other prediction horizons and 
accuracy measures is anticipated from the analysis of 
selected rushes in sections 4.1 and 4.2 and confirmed 
by the additional summary diagrams in figures 11,12 
and 13. These figures summarize 5 diagnostic mea- 
sures of prediction accuracy, calculated at 3 horizons 
during the 36 rush periods. The plot format follows 
figure 9, but only the 9 day average at each rush time 
is shown, and, following the convention established in 
section 4.1, arrival time accuracy is calculated as ON- 
E/ON for airline predictions, ON-ETA for CTAS pre- 
dictions at the 15 min horizon and ON-STA for CTAS 
predictions at the 30 and 60 min horizons. A supple- 
mentary numerical table (table 2) lists averages over 
the set of 4 rush times, together with standard devia- 
tions relative to that average. 


The first row of plots in figure 1 1 summarizes val- 
ues for the arithmetic means of airline and CTAS pre- 
diction accuracy distributions at horizons 15, 30 and 
60 min from landing. An obvious feature of this figure 
is the similarity of the airline and CTAS data points: 
mean accuracy values are nearly identical at most rush 
times and show similar timewise variation. At the 60 
min horizon, predominantly positive accuracy values 
indicate that many flights arrive later than predicted. 
This result is expected because delay is often imposed 
closer to landing and not anticipated 60 min ahead. At 
the 30 min horizon, the prediction accuracy has pre- 
dominantly negative sign, reflecting the tendency for 
controllers to “frontload” the TRACON at the start of 
the rush, sending in some aircraft ahead of schedule to 
fill up available slots in the landing sequence. By the 
1 5 min horizon, most assigned delay has been 
absorbed, and predictions are generally accurate to 
within 30 sec. Averaged over all aircraft, these trends 
have such comparable effect on the airline and CTAS 
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Figure 1 1 . Mean and standard deviation of AAL and CTAS arrival time accuracy distributions, averaged over 9 test days. 
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Figure 12. Fraction of AAL and CTAS arrival time predictions accurate to within 2 and 5 min, averaged over 9 test days. 
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Figure 1 3. Fraction of AAL and CTAS arrival time predictions inaccurate by more than 10 min, averaged over 9 test days. 





Table 2. Time-averaged Measures of Prediction Accuracy 


Mean 




Prediction Horizon 



15 min 

30 min 

60 min 

AAL 

0.21 ± 0.85 

0.04 ± 1.04 

0.73 ± 1.41 

CTAS 

0.28 ± 0.40 

-0.34 ±0.72 

0.83 ±1.04 


Standard Deviation 




Prediction Horizon 



15 min 

30 min 

60 min 

AAL 

3.96 ± 1.16 

4.671090 

5.731 1.67 

CTAS 

1.25 ±0.29 

2.3910.52 

3.8910.83 | 


Fraction within ± 2 min 




Prediction Horizon 



15 min 

30 min 

60 min 

AAL 

0.5710.06 

0.51 ±0.08 

0.43 ± 0.08 

CTAS 

0.89 1 0.08 

0.66 ± 0.09 

0.41 ±0.10 


Fraction within ± 5 min 




Prediction Horizon 



15 min 

30 min 

60 min 

AAL 

0.88 1 0.05 

0.8210.06 

0.7310.10 

CTAS 

1.0010.01 

0.95 1 0.04 

0.81 10.10 


Number of outliers >10 min 




Prediction Horizon 



15 min 

30 min 

60 min 

AAL 

1.1 ± 1.2 

2.8 ±2.1 

4.5 ± 4.0 

CTAS 

0.1 ±0.3 

0.4 ±0.5 

1.3 ± 1.9 
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arrival time predictions, that an analysis based on the 
arithmetic mean accuracy measure reveals no particu- 
lar advantage for either set of values. 

A more definitive parameter for comparing airline 
and CTAS arrival time statistics is provided by the 
standard deviation of the prediction accuracy distribu- 
tions. Plots summarizing this measure for all rushes 
and prediction horizons are displayed in the second 
row of figure 1 1 , with the plot at the 30 min horizon 
duplicating data shown in figure 10. As noted in the 
discussion of figure 10, the standard deviation mea- 
sures the spread in accuracy values with an algorithm 
especially sensitive to the effect of “outlier” points. In 
figure 1 1, the CTAS data points lie consistently below 
the corresponding airline points, indicating superior 
CTAS predictive accuracy at all rush and horizon 
times. Moreover, the improvement with CTAS is seen 
to be greatest for predictions made close to the time of 
landing. This qualitatively evident trend may be quan- 
tified with the data in table 2, from which a simple cal- 
culation shows that the ratio of CTAS to airline 
standard deviation values decreases from 0.7 to 0.3 as 
the horizon decreases from 60 to 15 min. 

The greater precision of the CTAS arrival time 
predictions is again evident from the plots in figure 12, 
which show the fraction of predictions accurate to 
within 2 and 5 min. As in previous diagrams, the 
improvement with CTAS is most obvious at the 
smaller horizon times. At the 60 min horizon, the 
CTAS advantage is minimal and seen only in the 
slightly larger fraction of predictions accurate to 
within 5 min. Table 2 shows this fraction to be 81% 
for CTAS versus 73% for the airline, averaged over all 
rush periods. At the 30 min horizon, CTAS predictions 
had an average accuracy of 66% within 2 min and 
95% within 5 min, versus the accuracy of 51% and 
82% achieved by the airline. By the 15 min horizon, 
CTAS predicted 89% of all flights within 2 min and a 
full 100% within 5 min, a dramatic improvement over 
the fractions 57% and 88% achieved by the airline. 

A final set of summary plots in figure 13 illus- 
trates the potentially most valuable advantage for 
CTAS predictions: a significant reduction in the num- 
ber of “outlier” flights with arrival time errors greater 
than or equal to 10 min. These plots reveal a distinct 
improvement with CTAS at all rush and horizon 


times, but in this case the greatest benefit occurs at the 
60 min horizon. At that time, the airline predictions 
had almost 5 outliers per rush period, whereas with 
CTAS the number of large magnitude errors was 
reduced to approximately one. 

The above analyses using a variety of accuracy 
measures to contrast CTAS and airline arrival time 
predictions compellingly demonstrate the benefits 
derived from CTAS scheduling information, espe- 
cially during heavy rush periods and for those flights 
most difficult to forecast by the airline’s traditional 
methods. This improved predictive ability should have 
considerable value in minimizing the disruptive 
effects of miscalculated delay on airline operations. 

5. Concluding Remarks 

An analysis of data recorded during an experimen- 
tal test of the airline-CTAS information exchange sys- 
tem has demonstrated the utility of CTAS air traffic 
management advisory software in predicting accurate 
aircraft arrival times. Compared with traditional air- 
line arrival prediction, CTAS more precisely forecasts 
ATC-imposed delay. Especially significant is the 
reduction in prediction errors of 10 or more minutes, 
which could invoke large disruptions in airline hub 
and spoke operations. Furthermore, whereas conven- 
tional airline schedules are not updated beyond about 
30 minutes from landing, CTAS’s predicted landing 
times become increasingly accurate as aircraft near the 
terminal. Finally, the benefits realized from CTAS are 
greatest during periods of heavy traffic and large 
delay, the very conditions that cause the most disrup- 
tion to an airline’s operational efficiency. 

Although the present research did not include a 
comprehensive analysis of operational benefits 
accrued by the airline, an assessment of benefits to 
American Airlines during the April 1998 CAP Display 
System testing has been given by Zelenka et al. 
(ref. 1). These authors report that, on several occasions 
during the test period, AAL was assisted in equipment 
move-ups and flight shuffling and in preventing diver- 
sions for flights into its DFW hub. Moreover, the 
improvements in arrival time prediction and strategic 
fleet arrival planning were accomplished with no 
adverse impact on FAA air traffic control operations. 
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